Timer-based eye-tracking

ABSTRACT

Aspects of the present disclosure describe systems, methods, and structures that provide eye-tracking by 1) steering a beam of light through the effect of a microelectromechanical system (MEMS) onto a surface of the eye and 2) detecting light reflected from features of the eye including corneal surface, pupil, iris—among others. Positional/geometric/feature/structural information pertaining to the eye is determined from timing information associated with the reflected light.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication Ser. No. 62/611,477 filed 28 Dec. 2017 which is incorporatedby reference as if set forth at length herein. In addition, thisapplication includes concepts disclosed in United States PatentPublication No. 2016/0166146 published 16 Jun. 2016 and United Statespatent Publication No. 2017/0276934 published 28 Sep. 2017, each ofwhich is incorporated by reference as if set forth at length herein.

TECHNICAL FIELD

This disclosure relates generally to human—computer interfaces and morespecifically to eye-tracking systems, methods and structures thatadvantageously provide real-time measurements of eye-tracking and eyefixations.

BACKGROUND

As is known by those skilled in the art, human—computer interfaces areexpected to take advantage of visual input mechanisms includingeye-tracking mechanisms—resulting from a current trend in the emergingVirtual and Augmented Reality (VR/AR) enterprise

Of additional note, such eye-tracking mechanisms are expected to findwidespread applicability in medical ophthalmology, behavioralpsychology, and consumer measurement fields as well.

Given such applicability and importance, improved eye-tracking systems,methods and/or structures would represent a welcome addition to the art.

SUMMARY

An advance in the art is made according to aspects of the presentdisclosure directed to systems, methods, and structures providingtimer-based eye-tracking that advantageously facilitate a seamless,intuitive, non-invasive, interactive user interface between that userand smart devices including computers.

In addition to such human-computer interactions, timer-basedeye-tracking systems, methods and structures according to aspects of thepresent disclosure advantageously facilitate the development ofophthalmological measurement instruments for determining geometricand/or other eye features exhibiting a precision and reproducibilityunknown in the art. Such determinations advantageously include shape(s),geometry(ies), of eye feature(s) including the cornea, iris, sclera,etc., as well as their respective interfaces.

Finally, systems, methods, and structures providing timer-basedeye-tracking according to aspects of the present disclosureadvantageously facilitate the measurement of subject eye-movementsduring—for example—psychological or consumer behavior studies andevaluations.

In a broad context, systems, methods, and structures according to thepresent disclosure provides eye-tracking by 1) steering a beam of lightthrough the effect of a microelectromechanical system (MEMS) operatingat or near a resonant frequency, onto eye structures such as cornealsurface, iris, and/or sclera; and 2) detecting—by one or more discretedetectors (i.e, 4, 6, 8, . . . etc.)—the light reflected from thecorneal surface.

According to aspects of the present disclosure, a tracked glint (i.e.,short flash of light) may be detected as large amplitude pulses ofnarrow width whereas a tracked pupil will produce an absence ofreflected light in a region of a scanned pattern. Advantageously, one ormore discrete detectors may be selected to use a negative threshold forpupil tracking and/or a positive threshold for glint trackingthereby—and advantageously—enabling the discrimination between glintfeatures and pupil features.

Of further advantage—and according to still further aspects of thepresent disclosure—signals produced and output from a plurality ofdetectors may be summed such that multiple glints may be detected by asingle signal chain or that the effects of a non-uniform path lengthand/or lensing are equalized.

Advantageously—and according to aspects of the present disclosure, sinceall required relevant information is included in timing informationreceived from the one or more discrete detectors and any timing ofproduced pulses, a projected pattern (Lissajous) is employed thatadvantageously produces a superior pulse density over a projected regionof the eye. Of further advantage, when sufficient number of pulses aredetected/collected by the multiple detectors, a contour of the glint andlocation(s) of eye features may be obtained.

BRIEF DESCRIPTION OF THE DRAWING

A more complete understanding of the present disclosure may be realizedby reference to the accompanying drawing in which:

FIG. 1 is a schematic block diagram showing an illustrative eye trackingsystem according to aspects of the present disclosure;

FIG. 2(A) is a schematic diagram showing an illustrative geometricarrangement of the system of FIG. 1 according to aspects of the presentdisclosure;

FIG. 2(B) is a schematic diagram showing illustrative scan regions forarrangements such as those depicted in FIG. 2(A) and others according toaspects of the present disclosure;

FIG. 3 is a flow diagram showing an illustrative eye tracking methodaccording to aspects of the present disclosure;

FIG. 4 is a plot of a simple output pulse according to aspects of thepresent disclosure;

FIG. 5 shows a series of plots a plot of a simple output pulse accordingto aspects of the present disclosure;

FIG. 6 shows a series of plots illustrating pulse-width profiles takenfor an eye in two different positions according to aspects of thepresent disclosure;

FIG. 7 shows a plot illustrating noise-rejection double samplingaccording to aspects of the present disclosure;

FIG. 8 shows a schematic diagram illustrating a Lissajous curve patternas may be projected onto a surface of an eye and a more optimizedLissajous curve pattern according to aspects of the present disclosure;

FIG. 9(A) is a schematic diagram depicting an illustrative frame(goggle) having multiple photodiode detectors and a scanner disposedthereon and according to aspects of the present disclosure;

FIG. 9(B) is a schematic diagram depicting a top view of theillustrative frame (goggle) having multiple photodiode detectors and ascanner disposed thereon of FIG. 9(A) and according to aspects of thepresent disclosure;

FIG. 9(C) is a schematic diagram depicting a perspective view of anillustrative eyeglass frame having multiple photodiode detectors and ascanner disposed thereon and according to aspects of the presentdisclosure;

FIG. 9(D) is a schematic diagram depicting a top view of theillustrative eyeglass frame having multiple photodiode detectors and ascanner disposed thereon of FIG. 9(C) and according to aspects of thepresent disclosure;

FIG. 9(E) is a schematic diagram depicting a front view of theillustrative eyeglass frame having multiple photodiode detectors and ascanner disposed thereon of FIG. 9(C) and according to aspects of thepresent disclosure;

FIG. 9(F), FIG. 9(G) and FIG. 9(H) are plots resulting from simulationresults identifying photodiode detector coverage as a function of eyepitch vs eye yaw for: FIG. 9(F)—an IPD of 63 mm; FIG. 9(G)—an IPD of 59mm; and FIG. 9(H)—an IPD of 70 mm; all according to aspects of thepresent disclosure;

FIG. 10 is a plot of pulses for clock synchronization in multi-clockconfigurations in terms of its delay with respect to a local X and Ytimer that is set to frequencies near the drive frequencies according toaspects of the present disclosure;

FIG. 11 is a flow diagram showing one illustrative method for Lissajousprojection optimization according to aspects of the present disclosure;

FIG. 12 is a flow diagram showing one illustrative method for ellipsefitting according to aspects of the present disclosure; and

FIG. 13 is a flow diagram showing one illustrative method for pupiltracking according to aspects of the present disclosure;

The illustrative embodiments are described more fully by the Figures anddetailed description. Embodiments according to this disclosure may,however, be embodied in various forms and are not limited to specific orillustrative embodiments described in the drawing and detaileddescription.

DESCRIPTION

The following merely illustrates the principles of the disclosure. Itwill thus be appreciated that those skilled in the art will be able todevise various arrangements which, although not explicitly described orshown herein, embody the principles of the disclosure and are includedwithin its spirit and scope.

Furthermore, all examples and conditional language recited herein areprincipally intended expressly to be only for pedagogical purposes toaid the reader in understanding the principles of the disclosure and theconcepts contributed by the inventor(s) to furthering the art, and areto be construed as being without limitation to such specifically recitedexamples and conditions.

Moreover, all statements herein reciting principles, aspects, andembodiments of the disclosure, as well as specific examples thereof, areintended to encompass both structural and functional equivalentsthereof. Additionally, it is intended that such equivalents include bothcurrently known equivalents as well as equivalents developed in thefuture, i.e., any elements developed that perform the same function,regardless of structure.

Thus, for example, it will be appreciated by those skilled in the artthat any block diagrams herein represent conceptual views ofillustrative circuitry embodying the principles of the disclosure.Similarly, it will be appreciated that any flow charts, flow diagrams,state transition diagrams, pseudo code, and the like represent variousprocesses which may be substantially represented in computer readablemedium and so executed by a computer or processor, whether or not suchcomputer or processor is explicitly shown.

The functions of the various elements shown in the Drawing, includingany functional blocks that may be labeled as “processors”, may beprovided through the use of dedicated hardware as well as hardwarecapable of executing software in association with appropriate software.When provided by a processor, the functions may be provided by a singlededicated processor, by a single shared processor, or by a plurality ofindividual processors, some of which may be shared. Moreover, explicituse of the term “processor” or “controller” should not be construed torefer exclusively to hardware capable of executing software, and mayimplicitly include, without limitation, digital signal processor (DSP)hardware, network processor, application specific integrated circuit(ASIC), field programmable gate array (FPGA), read-only memory (ROM) forstoring software, random access memory (RAM), and non-volatile storage.Other hardware, conventional and/or custom, may also be included.

Software modules, or simply modules which are implied to be software,may be represented herein as any combination of flowchart elements orother elements indicating performance of process steps and/or textualdescription. Such modules may be executed by hardware that is expresslyor implicitly shown.

Unless otherwise explicitly specified herein, the FIGs comprising thedrawing are not drawn to scale.

As will become apparent to those skilled in the art, systems, methods,and structures according to aspects of the present disclosureadvantageously extend the capabilities of gesture tracking systemsdisclosed in U.S. Patent Publication No. US2016/0166146 (hereinafterreferred to as the '146 publication), which disclosed scanningmicroelectromechanical systems that determine the position of an eye bydirecting a beam of light towards the eye and determining the uniqueangle at which the beam reflects off the cornea of the eye to determinethe direction of the gaze of the user. Systems in accordance with the'146 publication enable eye tracking that can be faster, lower power,more precise, and lower cost than prior-art video-based systems.

FIG. 1 shows a schematic block diagram illustrating an eye trackingsystem according to aspects of the present disclosure. As will beapparent to those skilled in the art by inspection of this figure andthe following discussions, such illustrative systems constructedaccording to aspects of the present disclosure advantageously exhibitsubstantial improvements in size, cost, power consumption, bandwidth andprecision as compared with prior art eye-tracking systems.

With reference to FIG. 1, illustrative system 100 includes one or moretransmit module(s) 102, detect module(s) 104 (that may include multipleindividual detectors)—not specifically shown, and processor(s) 106. Notethat for simplicity in the drawing, only single module(s) are shown inthis illustrative figure. Those skilled in the art will of courseappreciate that the number(s) and function(s) of the module(s) are notfixed, and instead may include a plurality of same. Still further, theirrespective position(s) may likewise be varied from those illustrativelyshown including spaced-apart relative to one another and/or arranged ina pre-determined or no particular arrangement around—forexample—eyeglass frames or goggles or shield or other mechanicalsupport.

Continuing with our discussion, transmit module 102 and detect module104 are illustratively shown arranged on a rigid support in a fixedorientation relative to an eye 120 of a test subject. As we shall showand describe, system 100 enables tracking of a surface feature (e.g.,cornea 124 or other feature including pupil, iris, sclera—notspecifically shown) within a two-dimensional region of an eye duringtypical test subject behavior (e.g., reading, viewing a computer screen,watching television, monitoring a scene, shopping, other consumeractivities, responding to stimulus, etc.), and estimating and/ordetermining the corneal vector of the eye based on the location of thesurface feature (and perhaps other characteristics).

For the purposes of this Specification, including the appended claims,the “corneal vector” or “gaze vector” of an eye is defined as the gazedirection of the eye. As may be readily appreciated by those skilled inthe art, we note that the optical axis of an eye is not the same as avisual axis. More specifically, the optical axis may be substantiallyaligned—for illustrative example—with an optical centerline of the eyewhile the visual axis is more substantially aligned with a visual acuitylocation of the eye, namely the fovea centralis. The fovea isresponsible for sharp central vision, which is necessary in humans foractivities where visual detail is of primary importance, such as readingand driving. Accordingly, a gaze vector is preferably indicated by avector extending outward along the visual axis. As used herein and aswill be readily understood by those skilled in the art, “gaze” suggestslooking at something—especially that which produces admiration,curiosity or interest—among other possibilities.

Transmit module 102 is a sub-system for providing an optical signal andscanning it in two-dimensions over a scan region 122 of eye 120.Transmit module 102 provides input signal 116, which is a light beamdirected at eye 120. Exemplary transmit modules are described in detailin the '146 publication; however, it should be noted that transmitmodules in accordance with the present invention are not limited tothose disclosed in the '146 publication.

Detect module 104 is a sub-system for receiving light reflected fromscan region 122, providing an electrical signal based on the intensityof the reflected light, and detecting—among other possible things—one ormore maxima in the electrical signal. Exemplary detect modules aredescribed in detail in the '146 publication; however, it should be notedthat detect modules in accordance with the present invention are notlimited to those disclosed in the '146 publication. As noted previously,while only a single detect module is shown in the illustrative FIG. 1,those skilled in the art will appreciate that more than one detectmodule may be employed—each having one or more individual detectorsincluded therein. As will become further appreciated, suchconfigurations including multiple detect modules and/or multipledetectors therein, provide additional data/information according toaspects of the present disclosure. We note at this point that the abovediscussion generally describes the detection of maxima. Advantageously,minima are also possible to detect in the case of the pupil. As we shallfurther show and describe, systems, methods and structures according toaspects of the present disclosure may detect/identify edges of featureswhich may be more difficult to identify. In such an application, we mayadvantageously edge outlines of features and fit—for example—ellipses tofacilitate their identification.

Continuing with our discussion of FIG. 1, processor 106 may be aconventional digital processor and controller (e.g., a microcontroller,microcomputer, etc.) operative for controlling transmit module 102,establishing system timing, and estimating the two-dimensional locationof cornea (for example) 124 within scan region 122. In the depictedexample, processor 106 communicates with transmit module 102 and detectmodule(s) 104 via wired connections (not shown) to transmit and receivecontrol signals 126 and output signal 128. In some embodiments,processor 106 communicates with transmit module 102 and detect module104 wirelessly. In some further embodiments, processor 106 is integratedin one of transmit module 102 and detect module(s) 104. Note furtherthat in those embodiments including multiple detector modules there maybe multiple output signal 128 lines communicating with processor. Notefurther that in those configurations including multiple detectorsincluded as part of a single detector module, the multiple detectors mayprovide individual, multiple signal lines to the processor as well ormay be locally processed by detector module thereby providing a singlesignal to the processor.

In the depicted, illustrative example, system 100 is mounted on eyeglassframes 108, which includes temples 110, lenses 112, and bridge 114.System 100 is shown mounted on frames 108 such that transmit module 102and detect module(s) 104 are on opposite sides of central axis 126 ofeye 120. Specifically, transmit module 102 is mounted on the frames suchthat it can scan input signal 116 over the full extent of scan region122 and detect module(s) 104 is/are mounted on the frames such thatit/they can receive a portion of input signal 116 reflected from scanregion 122 as reflected signal 118. As noted previously, one or moredetect module(s) may include one or more individual detectors which, aswe shall show and describe, advantageously provide enhanced performanceand informational value for systems, methods, and structures accordingto the present disclosure as compared with the prior art.

In particular, the specific location(s) of the one or more detectmodules including one or more individual discrete detectors may beadjustable on the frame structures such that systems, method, andstructures according to the present disclosure may advantageouslyprovide enhanced informational value for a larger portion of thepopulation. We note further that multiple detect modules and or multipledetectors advantageously improve robustness, more accurate eye profiles,geometry determinations, in addition to the improved gaze direction dataalready noted.

FIG. 2(A) is a schematic diagram depicting an illustrative geometricarrangement for system 100. At this point that it is noted thataccording to one aspect of the present disclosure that there exists aconfiguration of system 100 that gives rise to a unique point on cornea124 that results in a maximum intensity in the reflection of inputsignal 116 (i.e., reflected signal 118) at detector(s) 204 of detectmodule(s) 104, where detector(s) 204 is/are a discrete detector. For thepurposes of this disclosure, including the appended claims, a “discretedetector” is defined as an optoelectronic device having no more thanfour electrically independent detection regions on a single substrate,where each detection region is operative for providing one electricalsignal whose magnitude is based on the intensity of light incident uponthat detection region. Examples of discrete detectors include detectorshaving only one detection region, split detectors having two detectionregions, four-quadrant detectors having four detection regions, andposition-sensitive detectors. The definition of discrete detectorexplicitly excludes individual pixels, or groups of pixels, within arraydevices for collectively providing spatially correlated imageinformation, such as focal-plane arrays, image sensors, and the like.When input signal 116 is aligned with this point, the angular positionsof scanner 202 within transmit module 102 are indicative of the locationof this point of maximum reflection within scan region 122, which isindicative of the corneal vector for the eye.

As may be observed from FIG. 2(A), are position(s) of cornea 124 atthree gazing positions namely, (1) gazing straight ahead and alignedwith central axis 126, as indicated by cornea 124′ and corneal vectorCV′; (2) gazing in the extreme positive direction, as indicated bycornea 124″ and corneal vector CV″; and (3) gazing in the extremenegative direction, as indicated by cornea 124″′ and corneal vectorCV″′.

Turning now to FIG. 2(B), there is shown a schematic diagram depictingan exemplary scan region 122 of a subject eye. As illustrativelydepicted, scan region 122 extends from x=xMin to x=xMax and from y=yMinto y=yMax in the x- and y-directions, respectively.

During an illustrative operation of system 100, scanner 202 sweeps inputsignal 116 over scan region 122 in two dimensions. When the input signalis incident on cornea 124, reflected signal 118 (i.e., the cornealreflection) sweeps over detector 204. We note that during operation—andas we shall show an describe—the two dimensional scan may occursimultaneously i.e., it moves in both directions simultaneously and maybe projected onto the eye in a specific pattern to provide enhancedoperation. It should be noted that the curvature of the cornea givesrise to a reflective condition that reduces the angle-of-reflection to anarrow range of scanner angles. The position of the scanner thatcorresponds to the maximum received intensity at the aperture ofdetector 204 is then used to calculate the location of the cornea, whichis then used to estimate corneal vector CV.

As previously noted, the particular sweep of input signal mayadvantageously be shaped over scan region(s) such that a desired sweepdensity is achieved thereby producing a desirable (i.e., greatest)density of received pulses produced by the one or more discretedetectors. While the particular sweep shape is user definable, oneparticular shape—the Lissajous—produces a surprisingly effective sweepand therefore pulse densities.

Those skilled in the art will appreciate that a Lissajous curve—alsoknown as a Lissajous figure—is the graph of a system of parametricequations defined by x=A sin(at+δ); y=B sin(at). In one exemplaryoperation, the sweep is performed such that the rate of change of the xcoordinate is substantially the same as the rate of change of the ycoordinate.

We note at this time that a typical human eye has an eyeball diameter ofapproximately 24 mm and a cornea having a 9 mm radius of curvature, withthe cornea projecting a few millimeters above the eye, thereby defininga surface feature. Based upon this typical eye configuration, in thedepicted illustrative example, transmit module 102 and detect module 104are shown positioned symmetrically about central axis 126 at half-width,W, (half the normal distance across a typical eyeglass lens) ofapproximately 25 mm. Vertex line 128 is a straight line connecting thecenter of scanner 202 and the center of the aperture of detector 204.Vertex distance, D, (i.e., the distance between vertex line 128 and theapex of cornea 124 when the eye is aligned with central axis 126) isselected as approximately 14 mm. The locations of transmit module 102and detect module(s) 104 are selected to substantially maximize therange over which a reflected signal 118 is received for all corneallocations within scan region 122.

We note that the configuration/distances described above are only forillustrative purposes only. As will be readily appreciated, theparticular geometries/feature location(s) will vary from individual toindividual. As such, systems, methods, and structures according to thepresent disclosure do not necessarily require the particularconfigurations noted. In particular, a scanner may be located in mostany location providing optical paths to the surface of the eye and itsscan region to be scanned. As such, the scanner may be positioned on thenose side, the bottom, or the outside—or practically any location—of thesupport frame. When so positioned, the multiple detector elements may bepositioned circumferentially around the frame, or in other locationsfrom which suitable reflections may be detected.

We note that the '146 publication referenced previously, discloses amethod for determining eye position (i.e., corneal position) when areflected light signal exhibits a maximum intensity. In contrast,systems, methods, and structures according to the present disclosureadvantageously employ tracking methodologies that may advantageouslyinclude one or more of the following: pulse-width tracking; leading-edgetracking; last-edge tracking; modulated tracking; and noise-rejectingdouble sampling.

Pulse-Width Tracking Method

Turning now to FIG. 3, there is shown a flow diagram depicting thegeneral operations of a pulse-width tracking method in accordance withaspects of the present disclosure. We note that such methodadvantageously may be employed to produce pulse width informationregardless of scan direction to effectively determine glint/pupil/othereye feature information. We describe method 300 herein with simultaneousreference to FIG. 1, FIG. 2(A), and FIG. 2(B), as well as reference toFIG. 4, and FIG. 5.

Returning to FIG. 3, pulse-width tracking method 300 begins atoperational step 301, wherein a threshold 402 is established. We notethat such threshold is advantageously user definable. By appropriatelyadjusting the threshold, systems, method, and structures according tothe present disclosure—in addition to identifying glints—mayadvantageously identify quite subtle eye features such as interfacebetween iris and pupil or other feature(s). Those skilled in the artwill of course understand and appreciate that the identification of suchsubtle features—particularly during movement of an eye—has proven quitedifficult to achieve in the art. Advantageously, systems, methods, andstructures according to the present disclosure may effectively overcomesuch difficulties and provide informational value heretofore unknown inthe art.

Continuing with our discussion of method 300 illustratively outlined inFIG. 3, at step 302, a reflected signal 118 is detected at detectormodule 104. Note that a pulse-width method for eye tracking according toaspects of the present disclosure advantageously exploits therelationship between a signal amplitude and a total signal time. Wheninput signal 116 is scanned in a given direction—preferably at aconstant rate—a reflected signal 118 is received at a detect module 104,which in turn provides output signal 126 in the form of a detectionpulse, which appears for a finite time. The amount of time that theintensity of reflected signal 118 exceeds a pre-determined fixedthreshold is proportional to the amplitude of the detection pulse.

Such operation may be understood with simultaneous reference now to FIG.4. which shows a simplified plot of a representative detected pulseaccording to aspects of the present disclosure. As may be observed fromthat figure, plot 400 shows the amplitude of output signal 126 relativeto threshold 402 as a function of time, t. As depicted in plot 400,output signal 126 takes the shape of pulse 404, which crosses threshold402 in the rising direction at time, t1, and crosses threshold 402 inthe falling direction at time, t2. The pulse width of pulse 404 (i.e.,pulse width 406), therefore, is equal to t2−t1.

Continuing our discussion of the flow diagram of FIG. 3, at operation303, for each of i=1 through N, where N has any practical integer value,input signal 116 is scanned in a predetermined pattern about scanregion. As those skilled in the art will readily appreciate, asimplified scan operation may proceed horizontally from xMin to xMax ata different y-position, yi, on eye 120 in a manner analogous to a rasteroperation. Of course—and as noted throughout this disclosure—systems,methods, and structures according to aspects of the present disclosureadvantageously do not proceed in such a rasterized manner. In sharpcontrast, systems, methods, and structures according to the presentdisclosure will scan a pattern such as the Lissajous previouslyidentified and in that advantageous manner will produce a scan/outputsignal density that is surprisingly much more effective that thesimplified raster.

At step 304, for each of i=1 through N, pulse 404-i is generated bydetector module 104 and its pulse width 406-i is determined. While notspecifically shown in the flow diagram, those skilled in the art willunderstand and appreciate that the detection may be accomplished by oneor more detector modules each including one or more individual detectorelements.

FIG. 5 shows a series of plots depicting pulse-width tracking inaccording to aspects of the present disclosure. Note that in thisspecific example, the tracking is illustratively performed in thevertical direction. As shown in that figure, plots 500, 502, and 504show pulses 404 taken at three different y-positions—namely, y=1, 0, and−1, respectively.

With reference once again to FIG. 3 flow diagram, at step 305, a profileof pulse widths is generated based on pulses 404-1 through 404-N. Atstep 306, a maximum pulse width, b, is determined from the profile ofpulse widths.

FIG. 6 shows a series of plots depicting pulse-width profiles taken foran eye in two different positions according to aspects of the presentdisclosure. As may be observed from that figure, plot 600 includespulse-width profiles 602 and 604, which are taken while eye 120 is in afirst and a second orientation(s). In the illustratively depictedexample, these two orientations are at the same x-position but atdifferent vertical positions. We note that points a, b, and c as shownin FIG. 6 correspond to the pulse-width values determined forx-direction scans at three y-positions on eye 120 when the eye is in thefirst orientation. Point “b” corresponds to the maximum pulse width forthis eye orientation, while points a and c correspond pulse width valuesfor scans taken at other y-positions on the eye.

Finally—with respect to FIG. 3 flow diagram—at step 307, the y-positionassociated with the maximum pulse width is determined.

At this point we again note that systems, methods, and structuresaccording to the present disclosure may advantageously employ thepulse-width method and resulting derived information therefrom when anyscan direction is employed. Accordingly, this disclosure isadvantageously not limited to the simplified, x and y directionsillustratively employed for discussion purposes.

Returning once again to FIG. 6, we note that when eye 120 moves from itsfirst orientation to its second orientation, the pulse-width profileshifts upward, as indicated in the figure. In this second orientation,the peak width values for x-direction scans taken at the same threey-positions as above are indicated by a′, b′, and c′. It should be notedthat in this illustrative example, a small vertical movement of the eyegives rise to a large change in the measured difference in width betweenpoints a′ and c′ relative to the initial difference in width between aand c. As a result, systems, methods, and structures according toaspects of the present disclosure will advantageously achieve a highsignal-to-noise ratio (SNR) by employing a controller that minimizes thewidth difference between a′ and b′.

Leading-Edge Tracking Method

Leading-edge vertical tracking exploits the relationship between signalamplitude and earliest signal time (i.e., t1). By way of illustrativeexample, when an input beam 116 is scanned horizontally across the eye(i.e., in the x-direction), a detection pulse appears on the outputsignal of detector 204 for a period of time. When compared to fixedthreshold 402, the earliest time the pulse appears above the threshold,t1, is proportional to the amplitude of the pulse. A profile of risingedges is generated by performing a plurality of x-direction scans atdifferent y-positions. Comparing the timing of these rising edgesenables the determination of an earliest rising edge, which can then beused for vertical tracking.

As previously noted, exemplary x, y, direction scans are used herein forillustrative purposes only. Accordingly, the tracking methods describedherein may advantageously be employed in systems, methods, andstructures according to aspects of the present disclosure regardless ofa specific scan direction.

Last-Edge Tracking Method

Last-edge tracking exploits the relationship between signal amplitudeand last signal time. When input beam 116 is scanned in the x-direction,a detection pulse appears on the output signal of detector 204 for atime. When compared to fixed threshold 402, the last time the magnitudeof the pulse is equal to or greater than the threshold (i.e., t2) isproportional to the amplitude of the pulse. As a number of x-directionscans are performed at different y-positions, a profile of falling edgesis generated. Comparing the timing of these falling edges allows for thedetermination of a last rising edge, t2, which is used for verticaltracking.

Modulated Tracking Method

According to aspects of the present disclosure, periodically turning thelight source in transmit module 102 on/off at a high frequencyadvantageously enables enhanced SNR in tracking. In such embodiments,detect module 104 includes a detector circuit (e.g., a phase-lockedloop, etc.) that is tuned to the on/off frequency of the source,enabling it to pick up the desired signal while rejecting interferingsignals.

Noise-Rejection Double-Sampling Method

Finally, we note that systems, methods, and structures according toaspects of the present disclosure may employ a noise-rejectiondouble-sampling technique in which low-frequency noise in output signal126 can be rejected by software that enables double sampling.

Turning now to FIG. 7 there is shown a plot illustrating noise-rejectiondoubling sampling according to aspects of the present disclosure. As maybe observed and as shown in plot 700, the baseline amplitude is measuredat the beginning and end of each scan. The peak amplitude during thescan is also measured. The amplitude of the peak relative to thebaseline (rather than the absolute amplitude of the peak) can bedetermined by comparing it to the baseline measurements. The relativepeak amplitude can be obtained by subtracting the first baselinemeasurement from the peak amplitude, or by subtracting the end baselinemeasurement from the peak amplitude. The relative peak amplitude canalso be obtained by interpolating the baseline amplitude at the peakusing the two baseline measurements. Using the timestamp of the peak andboth baseline measurements allows for a linear interpolation of thepeak's baseline amplitude. Higher order interpolations can be used forgreater accuracy if more previous baseline measurements are stored. Inthis way, low frequency noise that shows in the peak's absoluteamplitude becomes negligible when the relative amplitude is captured.Note further that in noise-rejection double sampling it may be useful toadjust laser power such that the detected power at the beginning and/orend of a cycle are at a prescribed level. this is related to opticalpower modulation as well. In optical power modulation one purpose is tonormalize for beam velocity, however here, we may also normalize foruniformity as well.

With these techniques in place, we may now describe some additional,particularly distinguishing aspects of systems, methods, and structuresaccording to the present disclosure.

Lissajous Scan Patterns

As noted previously, a preferred projected scan pattern includes afamily of curves commonly known as Lissajous curves. One characteristicof such curves that that they generally move sinusoidally in both axessimultaneously. Operationally—with systems, methods and structuresaccording to the present disclosure that may advantageously employ MEMSdevices, both x, and y axis of the MEMS are driven near their resonantfrequencies which advantageously results in enhanced power andmechanical range(s). Of further advantage, we have determined thatLissajous curves provide a superior scan pattern for a given surface, asit permits a very fast (fastest) scan speed for a given mechanicalsystem having mechanical bandwidth constraints. FIG. 8 illustrates arepresentative Lissajous curve pattern that may be projected onto asubject eye and a more optimized Lissajous pattern according to aspectsof the present disclosure.

As will be further understood an appreciated by those skilled in theart, not all Lissajous curves exhibit the same pattern density.Accordingly, when scanning to determine glints (eye features) one wantsto cover a scan area in a reasonably short period of time such that thefeature is also located within the reasonably short period of time. Aswill be understood, the specific density of pattern projected onto theeye is a function of the drive frequencies of the MEMS device employed.

Still, it is noted that during operation, if a particular glint isdetected and subsequently tracked, it may be possible to set/adjust a DCcomponent of the scanner to target or otherwise “zero in” on that glintthereby allowing the scanner to operate at substantially resonance in inone of the two dimensions. Such operation advantageously permits use ofmore sparse patterns thereby requiring tighter duration constraints andincreasing a report rate of glint tracking.

We note further that when tracking multiple glints, or when tracking ina manner that is tolerant of slip, the scanner may require a larger canrange such that all of the glints are sufficiently detected. In such asituation, the relative density of the projected pattern should behigher, as the individual glints are smaller with respect to theparticular size of the scan window employed. In such a circumstance, theduration constraint must be relaxed and the report rate of individualglints must be decreased.

Operationally, steps associated with an illustrative Lissajousprojection optimization are shown in flow diagram of FIG. 11 and may bedefined as follows:

-   -   1. Scan a horizontal line at a vertical offset that produces        pulses at one of several photodiodes;    -   2. Sweep the frequency of the signal that drives the horizontal        axis while measuring the pulse width received by the photodiode;    -   3. The resonant frequency of the MEMS scanner corresponds to the        frequency at which the pulse width is minimized. The slope of        the phase of the pulse with respect to the drive signal should        be at a maximum at this frequency as well. The phase should        shift by 180 degrees as the device sweeps through its resonance        as well;    -   4. Select a frequency that is offset from resonance (e.g. 10%        higher or lower);    -   5. Perform steps 1-4 for the vertical axis;    -   6. While running both axes at their selected frequencies,        measure the pulse density received by each of the photodiodes;        and    -   7. Fine tune the frequencies in order to maximize the pulse        densities received at the various photodiodes

At this point we note that one way of identifying the center of a glintor feature is to determine the pulsewidth and average the location ofall mid-points in that vicinity while performing a Lissajous scan. Sucha technique advantageously shown to be surprisingly effective whileexhibiting a lower computational cost than ellipse fitting. Furthermore,a center of mass may be created by weighting each mid-point by its pulsewidth, i.e., the larger the pulse width the greater the likelihood thatit is the a point that represents the center of the glint.

Panning Scan Pattern

Note that while a Lissajous pattern exhibiting a fixed extent isprojected towards an eye it is possible to adjust DC voltages in drivesignals to “pan” the pattern around the eye. If a glint is beingdetected while the pattern is being panned, its timing will be offset byan amount corresponding to the shift in the pattern. This timinginformation may be advantageously used to center the projected patternaround a glint or to position the pattern such that multiple glints arecaptured for a given eye (user) with a specific inter-pupil distance(IPD) and eye relief.

Elipse Fitting via Pulse Width Contour

As previously described, an output pulse is conveniently described—inthe context of systems, methods, and structures according to the presentdisclosure—as a duration in time between when a detector output signalexceed some predetermined threshold to when it falls below thatpredetermined threshold. In that sense, each pulse provides measurementof two locations (in time) namely when the pulse signal crosses thisthreshold on the rise and the fall. Therefore, if we know the scanpattern—that is the beam location as a function of time—then each pulseeffectively measures two spatial location where the signal is equal tothe threshold.

Accordingly, as the beam scans, we measure a number of locations wherethe signal is equal to the threshold. The set of these points is knownas the contour (like on a map) of the detector intensity at the giventhreshold. If the threshold is set such that a contour of a glint iscaptured, the pulses will form an ellipse shape over time. We note thatthe actual shape depends on scan beam shape, local corneal curvature,and photodetector shape—among others—but nevertheless, an ellipticalcontour results. We can then employ fitting functions on a sparse set ofthe contour points (say, all of the points gathered in a particularperiod of time—say 10 milliseconds) to fit an ellipse whichadvantageously provides a low-latency measurement of glint location.Still further, when one has identified a range of ellipse dimensionsthat may be encountered, those dimensions may be advantageously used todiscriminate between corneal glints and disturbances thereby enhancingany determinations made with respect to actual features.

Of course, those skilled in the art will appreciate that such techniquesfacilitate the capture/contouring of multiple glints on a same detectorthrough use multi-model-fitting techniques.

We note that it may be necessary to determine which detected glints areones to keep and which ones are to be rejected. One illustrativetechnique to reject outlier ellipses (RANSAC—Random Sample Consensus)involves choosing a random subset of the ellipses, predict whichdirection they should be moving and apply that predicted direction tothe remaining ellipses. The one(s) that do not fit the model getrejected. An illustrative method of ellipse fitting is depicted in aflow diagram of FIG. 12 and may be described as follows:

-   1. For a given photodiode, collect rising and falling edge timings    for a period of time;-   2. Fit an ellipse to the contour defined by these points;-   3. Use an algorithm (e.g. RANSAC) to reject outlier ellipses;-   4. Relate some aspect of the geometry of the ellipse (e.g. its'    center, size, orientation) to a glint location, which serves as an    input to the eye model and;-   5. Using multiple ellipses from multiple photodiodes, use the eye    model to produce an estimate of the gaze direction.    Pupil Tracking (Pupillometry)

As we have noted previously, systems, methods, and structures accordingto aspects of the present disclosure may advantageously detectreflections resulting from other eye features/structures such as theedge of the pupil reflection as well as sclera reflections. As withcorneal reflections, such feature/structure reflections may be employedto determine gaze direction and tracking as well. We note that suchfeature/structure reflections may be quite subtle, and therefore anypulse width thresholds must be set sufficiently low so that signalsassociated with such features/structures are adequately detected andsubsequently identified.

Operationally, systems, methods, and structures according to aspects ofthe present disclosure will set a threshold at a predetermined pointsuch that edges of structures are detected and then determine the shapeof the pupil from the timings of threshold crossing in any (arbitrary)directions.

One illustrative method for pupillometry according to aspects of thepresent disclosure is shown in the flow diagram of FIG. 13. As shown,such method includes the following steps:

-   -   1. Measure the signal levels corresponding to specular glints        from the cornea, diffuse reflections from the iris, and the        lower signal levels (lack of iris reflection) from the pupil;    -   2. Set a threshold voltage for a comparator between the low        level signal from the pupil and the diffuse reflection signal        level from the iris;    -   3. Capture pulses that correspond to the pupil edge transitions        for a period of time and perform ellipse fitting technique        described above; and    -   4. Apply correction factors to compensate for the refractive        index of the cornea/lens and the direction in which the eye is        pointing in order to reveal the pupil size.

At this point we note that when attempting pupillometry, the signals arenot necessarily low as they are determined by the contrast from pupil toiris. The contrast is actually quite high—although orders of magnitudeless than a glint. One significant problem with pupillometry is that ofnon-uniform illumination/sensitivity across a scan range. In otherwords, pupillometry is negatively impacted by the non-uniformillumination wherein the path length between scanner and detector variesacross the scan range as reflected from the features of the eye. Anincreased path length drops the detected signal and therefore createsgradients that makes fixed threshold pupil detection difficult.Advantageously, and according to still further aspects of the presentdisclosure, one way to overcome this infirmity is to sum the signalsfrom multiple photodetectors such that the average path length of thebeam(s) is roughly equal as compared with any signal drop magnitudecreated by the pupil. Such summing may also be performed in a weightedmatter such that the signal is “leveled” against the background. Thiscalibration may occur—for example—when a user has their eyes closed soas to optimize a uniform diffuse reflection signal in the absence of thepupil thus making pupil detection easier.

We note further that systems, methods, and structures may advantageouslyadjust laser power dynamically to compensate for non-uniformillumination. In addition, the gain(s) or threshold(s) may bedynamically adjusted to mitigate the non-uniform illumination as well.

Optical Power Modulation with Scan Angle

Note that the amount of light received by an individual photodiodedepends upon its location with respect to the glint, and also withrespect to the portion of the Lissajous pattern that is illuminating theglint. If the corneal glint location is sufficiently close to thescanner but far from the photodiode, its brightness is lower due to thelensing effect of the cornea, which spreads/broadens the (originallycollimated) beam. Near the edges of the Lissajous pattern, the velocityof the scanned beam is lower, which implies that a spot “dwells” on thephotodiode for a longer period of time. As such, if a photodiodepreamplifier includes an integrating component, we may equalize areceived signal by modulating a source amplitude (i.e., VCSEL) withrespect to the velocity of the scanning beam.

According to still additional aspects of the present disclosure, sincethe timing of signals may be distorted by the sinusoidal trajectory ofthe beam, we may take the arcsin of any pulse timing to advantageouslyobtain linearized, position information.

Glint Rejection Using Multiple Detectors

Occasionally, a corneal glint as measured by one detector may “fall off”of the cornea and “turn into” a scleral glint. While this transition isoftentimes marked by a transient signal, if that transient signal is notnoticed or otherwise missed—with only one detector—it is difficult todetermine whether a glint is a corneal or scleral glint. When multipledetectors are employed—each having their own respective glintlocations—a geometric configuration may be configured such that it islikely most glints are in fact corneal. Advantageously, systems,methods, and structures according to aspects of the present disclosuremay employ such multiple detectors as noted previously and throughoutthis disclosure.

FIG. 9(A) is a schematic diagram depicting an illustrative frame(eyeglass goggles) having multiple photodiode detectors and a scannerdisposed thereon and according to aspects of the present disclosure. Asmay be observed from that illustrative figure, a single scanner (module)is shown along with a plurality of photodetector detectors/trackers (1-6in figure) disposed substantially around a perimeter of one of the eyes.Light directed from scanner to the eye will be advantageously reflectedand detected by the multiple detectors thereby providing enhancedinformation value. FIG. 9(B) is a schematic diagram depicting a top viewof the illustrative frame (eyeglass) having multiple photodiodedetectors and a scanner disposed thereon of FIG. 9(A) and according toaspects of the present disclosure.

Similarly, FIG. 9(C) is a schematic diagram depicting a perspective viewof an illustrative eyeglass frame having multiple photodiode detectorsand a scanner disposed thereon and according to aspects of the presentdisclosure. FIG. 9(D) is a schematic diagram depicting a top view of theillustrative eyeglass frame having multiple photodiode detectors and ascanner disposed thereon of FIG. 9(C) and according to aspects of thepresent disclosure. Finally, FIG. 9(E) is a schematic diagram depictinga front view of the illustrative eyeglass frame having multiplephotodiode detectors and a scanner disposed thereon of FIG. 9(C) andaccording to aspects of the present disclosure.

FIG. 9(F), FIG. 9(G) and FIG. 9(H) are plots resulting from simulationresults showing photodiode detector coverage as a function of eye pitchvs eye yaw for the arrangements of FIG. 9(A) in which FIG. 9(F)—an IPDof 63 mm; FIG. 9(G)—an IPD of 59 mm; and FIG. 9(H)—an IPD of 70 mm; allaccording to aspects of the present disclosure.

We note that if we make some reasonable assumptions about the sphericityof the cornea as a specular reflector, we can then construct a modelwhich compares the glints with each other to determine whether anindividual glint “disagrees” with other glints' locations. Such a modeladvantageously permits the detection of glints that do not appear to becorneal and can be therefore used to mark glints as something other thancorneal glints (i.e., non-corneal). Note however, that just because aglint does not lie on the cornea does not preclude that glint fromproviding useful information—particularly in the ophthalmologicalcontext noted previously.

Multiple Light Sources

As previously noted, multiple light sources may be advantageouslyemployed in systems, methods, and structures according to aspects of thepresent disclosure. As will be further appreciated by those skilled inthe art, measuring the distance of something is difficult if scanningfrom only a single perspective because distance has an inverserelationship with apparent size, and so has poor sensitivity acrossrange. If one employs two scanners, then two perspectives are createdthereby providing this additional information.

Multi-Clock Domain Synchronization

Finally, we note that when resonating at a given frequency, one expectsa detector to detect (see) two pulses for a given feature: one on theforward pass, and one on the reverse pass. In phase space (between 0 and360 degrees) this represents two distinct coordinates, the average ofwhich is the phase offset between the detector and the scanner. If thedetector does not know the phase of the scanner that is to say thescanner and detector are on separate clocks which may drift from eachother over time, the detector can determine the phase offset betweenitself and the scanner, by measuring a line of symmetry that separatesall the forward pass pulses and all the reverse pass pulses. With thephase offset known, it is then possible to reverse this phase offset,and convert the pulse from phase space, back into positional space whichmathematically amounts to running through an inverse trigonometricfunction).

When both axes of the scanner (for example—MEMS) are in resonance as isthe case during Lissajous scanning, we extend the same idea, except nowthere are 4 pulses for each feature (forward+up, forward+down,reverse+up, reverse+down). We then determine a line of symmetry in eachaxis, and this represents the phase offset in both axes—which can bedifferent, due to different mechanical properties of the two axes.

If we are constantly tracking the phase offsets over time as pulses aredetected, then we effectively are constantly synchronizing the clocksbetween the two systems, without having to perform any additionalsynchronization over other channels.

Operationally, illustrative steps for clock synchronization according toaspects of the present disclosure may be defined by:

First, capture pulse timings using one timer while projecting aLissajous pattern over the eye by driving a MEMS scanner with adifferent timer. Note, these timers are derived from different clocksources.

Next, plot each pulse in terms of its delay with respect to a local Xand Y timer that is set to frequencies near the drive frequencies asshown in FIG. 10. This effectively converts timer values into phasespace, since we know that the drive signal period corresponds to a full360 degrees. Note further that the scale of the x axis corresponds to afull cycle of the x-drive signal (360 degrees) and the scale of theyaxis corresponds to a full cycle of the y-axis drive signal (360degrees).

The resulting plot should produce 4 clusters of points for a givenphotodiode, which correspond to the beam sweeping across the detector ineach direction (forward+up, forward+down, reverse+up, reverse+down).Accordingly, compute the lines of symmetry between these clusters in thehorizontal and vertical directions.

Next, apply any necessary offsets in order to place these lines ofsymmetry in the center of the plot.

These offsets are then used to synchronize the receiving clock to theclock that is driving the MEMS scanner

At this point, while we have presented this disclosure using somespecific examples, those skilled in the art will recognize that ourteachings are not so limited. Accordingly, this disclosure should beonly limited by the scope of the claims attached hereto.

The invention claimed is:
 1. A timer-based eye-tracking methodcomprising: steering a beam of light through the effect of amicroelectromechanical system (MEMS), onto a surface of the eye; anddetecting light reflected from the surface; said method characterized inthat: the light steered onto the surface of the eye is steered such thatit is defined by a Lissajous curve; an electrical pulse is produced inresponse to the detected light; and the width of the pulse is used todetermine gaze direction.
 2. The method of claim 1 further characterizedin that: the Lissajous curve is described by the following relationshipsx=A sin(at+δ); y=B sin(bt) where a and b are natural numbers.
 3. Themethod of claim 1 further characterized in that: a gaze direction of theeye is determined from the detected reflected light.
 4. The method ofclaim 1 further characterized in that: one or more features of the eyeare determined from the reflected light.
 5. A timer-based eye-trackingmethod comprising: steering a beam of light through the effect of amicroelectromechanical system (MEMS), onto a surface of the eye; anddetecting light reflected from the surface; and producing electricalpulses in response to the detection; said method characterized in that:features of the eye are determined by widths of the pulses so produced.6. The method of claim 5 further characterized in that the beam of lightis steered such that it is defined by a Lissajous curve.
 7. The methodof claim 6 further characterized in that the Lissajous curve isdescribed by the following relationships x=A sin(at+δ); y=B sin(bt)where a and b are natural numbers.
 8. The method of claim 5 furthercharacterized in that a gaze direction of the eye is determined from thedetected reflected light.